Refining neural network predictions using background knowledge

Abstract Recent work has shown learning systems can use logical background knowledge to compensate for a lack of labeled training data. Many methods by creating loss function that encodes this knowledge. However, often the logic is discarded after training, even if it still helpful at test time. Instead, we ensure neural network predictions satisfy refining with an extra computation step. We introduce differentiable refinement functions find corrected prediction close original prediction. study how effectively and efficiently compute these functions. Using new algorithm called iterative local (ILR), combine refined formulas any complexity. ILR finds refinements on complex SAT in significantly fewer iterations frequently solutions where gradient descent not. Finally, produces competitive results MNIST addition task.